Estimating treatment effects from observational data is challenging due to two main reasons: (a) hidden confounding, and (b) covariate mismatch (control and treatment groups not having identical distributions). Long lines of works exist that address only either of these issues. To address the former, conventional techniques that require detailed knowledge in the form of causal graphs have been proposed. For the latter, covariate matching and importance weighting methods have been used. Recently, there has been progress in combining testable independencies with partial side information for tackling hidden confounding. A common framework to address both hidden confounding and selection bias is missing. We propose neural architectures that aim to learn a representation of pre-treatment covariates that is a valid adjustment and also satisfies covariate matching constraints. We combine two different neural architectures: one based on gradient matching across domains created by subsampling a suitable anchor variable that assumes causal side information, followed by the other, a covariate matching transformation. We prove that approximately invariant representations yield approximate valid adjustment sets which would enable an interval around the true causal effect. In contrast to usual sensitivity analysis, where an unknown nuisance parameter is varied, we have a testable approximation yielding a bound on the effect estimate. We also outperform various baselines with respect to ATE and PEHE errors on causal benchmarks that include IHDP, Jobs, Cattaneo, and an image-based Crowd Management dataset.

Data sources in contemporary machine learning applications are often heterogeneous, leading to potential distribution shifts Sugiyama and Kawanabe [2012], Shen et al. [2021]. This is a particularly relevant problem in computer vision Csurka [2017], healthcare Zhang et al. [2021], finance, Earth and climate sciences [Tuia et al., 2016, Kellenberger et al., 2021] and social sciences, as variations in data patterns can significantly impact model performance and generalisation in the out-of-distribution (OOD) setting, also referred to as domain generalisation [Shen et al., 2021, Zhou et al., 2023]. Various frameworks have been proposed to formally address the emergence of distribution shifts during the testing phase Peters et al. [2016], Arjovsky et al. [2020]. In cases where the data distribution is entailed by a Structural Causal Model (SCM) Peters et al. [2017], one can consider distribution shifts arising from intervention on specific variables of the SCM. Notably, the Instrumental Variable (IV) regression exhibits robustness to arbitrarily strong interventions [Bowden and Turkington, 1990]. However, pursuing algorithms robust to strong interventions may be overly conservative, especially when prior knowledge is available regarding the intervention strength that generates the distribution shift. Anchor Regression (AR) addresses this challenge by explicitly considering interventions on exogenous variables up to a specified strength [Rothenhäusler et al., 2018].

Most recommendation engines today are based on predicting user engagement, e.g. predicting whether a user will click on an item or not. However, there is potentially a large gap between engagement signals and a desired notion of "value" that is worth optimizing for. We use the framework of measurement theory to (a) confront the designer with a normative question about what the designer values, (b) provide a general latent variable model approach that can be used to operationalize the target construct and directly optimize for it, and (c) guide the designer in evaluating and revising their operationalization. We implement our approach on the Twitter platform on millions of users. In line with established approaches to assessing the validity of measurements, we perform a qualitative evaluation of how well our model captures a desired notion of "value".

By using the anchor variables for each item that needs to be predicted, the system will eventually learn the noise process from the latent variables (variables that are not directly observable), with the intent to eventually invert the process and undo the noise. Using this system, the assumption is that all of the anchor variables are conditionally independent from the other patient data. For instance, consider insulin as an anchor variable with diabetic patients. Whether it is or is not observed in a patient's medical records is unrelated to the other data observed for the patient.

Opinions expressed by Forbes Contributors are their own. The author is a Forbes contributor. The opinions expressed are those of the writer. Electronic Health Record (EHRs) systems are now used in 80% of doctors offices and contain a rich source of patient data available to innovate and improve healthcare. A team at New York University's Courant Institute of Mathematical Sciences developed algorithms and a system to extract EHR data to faster diagnose patients and provide a thorough understanding of the patient's health.